A system has been described whereby users of hypertext browsers may speak the hinks in hypertext documents using a dynamic-grammar, arbitrary-vocabulary, speaker-independent, continuous speech recognizer. See the following references: Hemphill, Charles T. and Thrift, Philip R., "Surfing the Web by Voice," Proceeding of Multimedia '95, No. 5-9, San Francisco, Calif. Philip R. Thrift and Charles T. Hemphill "Voice Activated World Wide Web Pages Using Grammatical Metadata (Smart Pages)", filed U.S. patent office Apr. 10, 1995, Ser. No. 08/419,229. A Protocol for adding natural language grammars to Web pages is described, which is utilized by a client-side speech recognition system to establish human to computer communication. This application is incorporated herein by reference. Charles T. Hemphill "Speaker-Independent Dynamic Vocabulary and Grammars in Speech Recognition", filed U.S. patent office: Apr. 10, 1995, U.S. Pat. No. 5,774,628. A method for dynamically creating grammars and pronunciations for arbitrary vocabulary tokens and how to incorporate them into a running, speaker-independent recognizer. This application is incorporated herein by reference. Increasingly, however, pages include more and more graphical images that do not explicitly indicate what the user might say to follow a link.
Other speech recognition systems have tried to cope with selection of graphical images by using speech commands to move the mouse. Such systems allow utterances such as "Mouse Move Up", Mouse Move Left", "Mouse Move Faster", "Mouse Move Slower", "Mouse Stop", "Mouse Click", etc. This method requires many queries and involves waiting for the mouse cursor to travel to the desired spot.
Another method divides the screen into grid areas (for example, 3 by 3) and lets the user position the mouse by selecting progressively more detailed areas. An interaction in this case might include utterances such as "Enable Grid", "Grid Area 5" (the grid reconfigures for the smaller area), "Grid Area 1" (the grid again reconfigures), "Grid Area 3", "Mouse Click". This method also requires many queries, but eliminates the wait for mouse cursor movement.